Skip to content

feat(taskprocessing): add worker command for synchronous task processing#59015

Merged
marcelklehr merged 8 commits intomasterfrom
copilot/add-taskprocessing-worker-command
Mar 19, 2026
Merged

feat(taskprocessing): add worker command for synchronous task processing#59015
marcelklehr merged 8 commits intomasterfrom
copilot/add-taskprocessing-worker-command

Conversation

Copy link
Contributor

Copilot AI commented Mar 17, 2026

Adds a taskprocessing:worker CLI command that runs a dedicated loop processing tasks via synchronous (ISynchronousProvider) providers, with starvation prevention via a single cross-type getNextScheduledTask query.

Summary

New command: occ taskprocessing:worker

  • Polls for and processes tasks from all preferred synchronous providers
  • --timeout <s>: exit after N seconds (default: run forever)
  • --interval <s>: sleep between idle polling cycles (default: 1s)
  • --once: process at most one task then exit
  • --taskTypes <id>: restrict to specific task type IDs (repeatable)

Starvation prevention

Instead of shuffling providers and issuing one getNextScheduledTask call per provider, all eligible task type IDs are collected first and passed in a single query. This guarantees the globally oldest scheduled task is always picked next, regardless of per-provider queue depth:

// Build eligible taskTypeId → provider map for all preferred sync providers
$eligibleProviders = [];
foreach ($providers as $provider) { ... }

// One query across all types — picks oldest task, preventing starvation
$task = $this->taskProcessingManager->getNextScheduledTask(array_keys($eligibleProviders));
$provider = $eligibleProviders[$task->getTaskTypeId()];
$this->taskProcessingManager->processTask($task, $provider);

Test fixes

  • createTask() helper now accepts an optional task type so task type IDs match provider expectations
  • testProcessesFirstMatchingProvidertestProcessesCorrectProviderForReturnedTaskType: updated mock expectations to reflect that getPreferredProvider is called for every eligible provider and getNextScheduledTask receives all eligible type IDs
  • Added testPicksOldestTaskAcrossMultipleEligibleProviders: verifies that with two eligible providers the task returned by getNextScheduledTask determines which provider runs

TODO

  • ...

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Add OCC command for task processing worker Add occ taskprocessing:worker dedicated synchronous task processing command Mar 17, 2026
Copilot AI requested a review from marcelklehr March 17, 2026 18:37
Copilot AI changed the title Add occ taskprocessing:worker dedicated synchronous task processing command Add --taskTypes whitelist option to taskprocessing:worker Mar 17, 2026
Copilot AI changed the title Add --taskTypes whitelist option to taskprocessing:worker taskprocessing:worker: add --taskTypes whitelist option Mar 17, 2026
@szaimen
Copy link
Contributor

szaimen commented Mar 18, 2026

Hey, is this the implementation for #52629 or is this something else?

Copilot stopped work on behalf of marcelklehr due to an error March 18, 2026 17:23
@marcelklehr
Copy link
Member

@szaimen This is something else. We discovered that the way TaskProcessing\SynchronousBackgroundJob works has some flaws that prevent seamless parallelism. So, for taskprocessing we've opted to leave the background job realm entirely and create a new command that runs the tasks in the background. (The alternative, in our view, would be to have one job per task which seems excessive.)

#52629 is separate from this effort and I realize that this PR distances us further from a unified solution. I'll pick up #52629 in our team discussion next week.

Copilot AI changed the title Implement occ taskprocessing:worker command to run tasks feat(taskprocessing): add worker command for synchronous task processing Mar 18, 2026
@marcelklehr marcelklehr force-pushed the copilot/add-taskprocessing-worker-command branch 2 times, most recently from cdd1d26 to b6812e1 Compare March 19, 2026 08:40
Copy link
Contributor

@kyteinsky kyteinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Copilot AI and others added 7 commits March 19, 2026 11:08
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…ing:worker command

Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…uffling providers

Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…ick the oldest task

Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
…tion-prevention test

Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
@marcelklehr marcelklehr force-pushed the copilot/add-taskprocessing-worker-command branch from b6812e1 to ad5e709 Compare March 19, 2026 10:08
Co-authored-by: Marcel Klehr <mklehr@gmx.net>
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
@marcelklehr
Copy link
Member

/backport to stable33

@marcelklehr
Copy link
Member

/backport to stable32

@marcelklehr marcelklehr merged commit 9e7e32f into master Mar 19, 2026
184 of 187 checks passed
@marcelklehr marcelklehr deleted the copilot/add-taskprocessing-worker-command branch March 19, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants